Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add linux audio screensharing #130

Merged
merged 27 commits into from
Oct 21, 2023
Merged

add linux audio screensharing #130

merged 27 commits into from
Oct 21, 2023

Conversation

Vendicated
Copy link
Member

@Vendicated Vendicated commented Sep 28, 2023

Adds audio screenshare on linux via pipewire

Allows you to choose to share either your entire system audio (without Discord), or a specific application

TODO

  • kill virtmic once stream is stopped

Needs testing

  • Behaves correctly on pulseaudio. That is: does not crash; displays "failed to retrieve audio sources"
  • works on fedora
  • works on ubuntu
  • works on steamos
  • works for sharing audio of individual apps
  • works for sharing system audio without including discord itself

resolves #117

@uku3lig
Copy link

uku3lig commented Sep 29, 2023

Doesn't work on wayland, the XDG portal window picker opens just fine and allows to select a window but then the vencord picker is empty:
image

In addition, pipewire seems to error after selecting something in the portal picker: 'loop->recurse > 0' failed at ../pipewire/src/pipewire/thread-loop.c:426 pw_thread_loop_wait()

Also after canceling clicking on the Go Live button again doesn't do anything, without anything being output in the logs and when closed exists with a SIGSEGV.

(Tested both on Hyprland 0.30.0 and GNOME 44.4)

@Vendicated
Copy link
Member Author

yes

@catgirlcataclysm
Copy link
Contributor

Doesn't work on wayland, the XDG portal window picker opens just fine and allows to select a window but then the vencord picker is empty: image

In addition, pipewire seems to error after selecting something in the portal picker: 'loop->recurse > 0' failed at ../pipewire/src/pipewire/thread-loop.c:426 pw_thread_loop_wait()

Also after canceling clicking on the Go Live button again doesn't do anything, without anything being output in the logs and when closed exists with a SIGSEGV.

(Tested both on Hyprland 0.30.0 and GNOME 44.4)

Try my fork, wayland-fixes branch.
Most of those issues should be resolved

@uku3lig
Copy link

uku3lig commented Sep 29, 2023

Try my fork, wayland-fixes branch. Most of those issues should be resolved

this seems to be working fine, except for one little issue: electron 26 turns the app into a white screen a few seconds after launch, as described in #125. works fine on electron 25.8.2

@maltejur
Copy link

maltejur commented Oct 4, 2023

Instead of the virtmic binary you could try to use https://github.com/kakxem/node-pipewire, that may be a bit cleaner than bundling a prebuilt binary.

@Vendicated
Copy link
Member Author

Vendicated commented Oct 11, 2023

these issues should now all be fixed (including the electron 26 whitescreen)


Instead of the virtmic binary you could try to use kakxem/node-pipewire, that may be a bit cleaner than bundling a prebuilt binary.

Curve is helping us turn it into a native addon instead. thanks for the idea though!

@maltejur
Copy link

Curve is helping us turn it into a native addon instead. thanks for the idea though!

Ah yeah that also sounds like a good idea.

I tested the code on KDE Wayland btw and it works great for me, besides the thing with the screen sharing portal popping up multiple times. That is a interesting bug, because it also happens in Chrome directly (when just trying to share your screen on a random website), but funnily enough not in qtwebengine (also based on chromium).

@Vendicated
Copy link
Member Author

besides the thing with the screen sharing portal popping up multiple times

yeah, it's sadly still an issue even after all these fixes. The first time it requests is normal, and it calls our electron registered handler. then right before it starts streaming, it asks again, this time without triggering the electron handler

on a random website

so it happens on any website, not just discord?

@maltejur
Copy link

so it happens on any website, not just discord?

Yup, on https://webrtc.github.io/samples/src/content/getusermedia/getdisplaymedia too for example. In Chrome it shows the portal, then the Chrome dialog, then the portal again, then the stream starts. In Firefox it works as expected, with it first showing the Firefox dialog, then the portal, and then starting the stream.

@Vendicated
Copy link
Member Author

what chromium version are you using? the site you linked works correctly for me on Version 117.0.5938.149 (Official Build, ungoogled-chromium) (64-bit)

@maltejur
Copy link

Version 118.0.5993.70 (Official Build) Arch Linux (64-bit)

Interesting, maybe this is a recent bug.

@maltejur
Copy link

maltejur commented Oct 11, 2023

Hm, no, the version ain't it. If I try the version you linked I get the same behaviour. My description of what is happening may have been a bit bad. At first the chrome dialog opens, when you try to select the "Window" tab, you get the first portal, and when you press "Share" you get the second portal.

@Vendicated
Copy link
Member Author

i just installed chrome and it works fine for me on Version 118.0.5993.70 (Official Build) (64-bit) (fedora)

here's a recording, in case i'm misunderstanding

Screencast.from.2023-10-12.00-17-14.webm

@Vendicated
Copy link
Member Author

Vendicated commented Oct 11, 2023

maybe it's an issue with pipewire itself or whichever DE/WM you use?
Here's mine:

gnome 44.5

pipewire
Compiled with libpipewire 0.3.81
Linked with libpipewire 0.3.81

or - considering it happens for me in electron too, maybe related to some chromium build flag?

@maltejur
Copy link

maltejur commented Oct 11, 2023

Pipewire is same for me, but I am using KDE

KDE Plasma Version 5.27.8
KDE Frameworks Version 5.110.0
Qt Version 5.15.11

pipewire
Compiled with libpipewire 0.3.81
Linked with libpipewire 0.3.81

I also tried the Chrome Flatpak, same behaviour. So I suspect this is more of a KDE/Gnome difference.

I'll try to attach a screen recording, but that is suprisingly difficult to create on Wayland :D

@maltejur
Copy link

2023-10-12.00-31-24.mp4

@Vendicated
Copy link
Member Author

i just installed kde and can't repro

webrtc.mp4

@Vendicated Vendicated marked this pull request as draft October 12, 2023 00:32
@Vendicated
Copy link
Member Author

Vendicated commented Oct 15, 2023

This pr is now very close to being done and you no longer need to build virtmic/venmic yourself

i have added a few things that still need testing to the description at the top. if you are able to test any of them, please do and report your results

@Vendicated Vendicated marked this pull request as ready for review October 17, 2023 03:41
@D3SOX
Copy link
Contributor

D3SOX commented Oct 17, 2023

It works for fine for me
image
pls merge

@D3SOX
Copy link
Contributor

D3SOX commented Oct 18, 2023

I just noticed this always exists after the audio selection input was shown once. Can't it be killed when no stream is running?
image

@Curve
Copy link
Member

Curve commented Oct 18, 2023

I just noticed this always exists after the audio selection input was shown once. Can't it be killed when no stream is running?
image

Recreating the virtual device would be absolutely unnecessary imho but could be added

@uku3lig
Copy link

uku3lig commented Oct 19, 2023

it seems that applications using pipewire-alsa are not detected (or at least don't show up in the audio selection menu), see attached screenshot (tested on pipewire 0.3.82, the two apps shown are osu-lazer launched with SDL_VIDEODRIVER=wayland and uku3lig/csf-reader)

edit: i dm'ed curve on discord about this issue

image

@Curve
Copy link
Member

Curve commented Oct 19, 2023

~snip

@uku3lig

Currently only applications that contain certain props are shown (namely application.process.id, application.process.binary and node.name).

Could you please send in the output of pw-dump while OSU is playing audio?

We might need to give the client control over which props he requires to be included (maybe an option similar to "Show all processes")

@Curve
Copy link
Member

Curve commented Oct 19, 2023

~snip

@uku3lig

Currently only applications that contain certain props are shown (namely application.process.id, application.process.binary and node.name).

Could you please send in the output of pw-dump while OSU is playing audio?

We might need to give the client control over which props he requires to be included (maybe an option similar to "Show all processes")

The output is probably going to be pretty big or may contain some personal info (username, and so on)

If you want to you can also DM me the output on discord: @curve.cpp

@wearrrrr
Copy link
Contributor

Just tested and it works flawlessly 👍, both app and entire system audio work. :D

@milenakos
Copy link

works good on ubuntu

@Vendicated
Copy link
Member Author

hi @uku3lig, could you try again now and see if your issue is fixed?

@uku3lig
Copy link

uku3lig commented Oct 21, 2023

works flawlessly, thank you very much!

@Vendicated Vendicated merged commit 573a953 into main Oct 21, 2023
1 check passed
@Vendicated Vendicated deleted the virtmic branch October 21, 2023 20:15
@afrozenpeach
Copy link

Vencord claims it can't retrieve my sound sources and says to use pipewire, but I'm definitely using pipewire. Here's a pw-dump output from my system.

pwdump.txt

@Vendicated
Copy link
Member Author

Vendicated commented Oct 26, 2023

probably related to using older glibc++. will be fixed once we publish our flatpak soon (and if you use that). Or you can update it manually

please don't necro closed prs in the future, instead make a new issue!

@leo60228
Copy link

leo60228 commented Nov 8, 2023

This might be a fairly niche usecase, but would it be possible to have a way to select a real audio input for screensharing? My usecase for this is streaming from a game console where my capture solution only handles video, so I just connect the console's audio output to my motherboard's line in jack.

@Curve
Copy link
Member

Curve commented Nov 8, 2023

This might be a fairly niche usecase, but would it be possible to have a way to select a real audio input for screensharing? My usecase for this is streaming from a game console where my capture solution only handles video, so I just connect the console's audio output to my motherboard's line in jack.

Venmic allows this, it would have to be utilized by vencord but that should be fairly easy

@Vendicated
Copy link
Member Author

Screenshot_2023-11-08-23-31-55-93_4d38fce200f96aeac5e860e739312e76

@Vencord Vencord locked as resolved and limited conversation to collaborators Nov 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Add screensharing with audio support for the linux app